Indicating out-of-view augmented reality images

ABSTRACT

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a see-through display system. For example, one disclosed embodiment includes identifying one or more objects located outside a field of view of a user, and for each object of the one or more objects, providing to the user an indication of positional information associated with the object.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 13/927,056, filedJun. 25, 2013, the entirety of which is hereby incorporated herein byreference for all purposes.

BACKGROUND

Augmented reality computing devices, such as head-mounted displaysystems and handheld mobile devices (e.g. smart phones, tabletcomputers, etc.), may be configured to display information to a userabout virtual or real objects in the field of view of the user and/or afield of view of a camera of the device. For example, a head-mounteddisplay may be configured to display via a see-through display systemvirtual information regarding a real-world object in spatialregistration with the real-world object. Similarly, a mobile device maydisplay such information via a camera viewfinder window.

SUMMARY

Embodiments are disclosed that relate to operating a user interface onan augmented reality computing device. For example, one disclosedembodiment includes identifying one or more objects located outside afield of view of a user, and for each object of the one or more objects,providing to the user an indication of positional information associatedwith the object.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B show schematic depictions of an example use environmentfor an augmented reality computing system.

FIG. 2 schematically shows an example embodiment of an augmented realitydisplay device.

FIG. 3 is a block diagram of the display device of FIG. 2.

FIGS. 4-8 show example indications of objects outside a field of view ofa user on an augmented reality display device.

FIG. 9 shows an example method for operating a user interface on anaugmented reality computing device in accordance with the disclosure.

FIG. 10 schematically shows an example computing system.

DETAILED DESCRIPTION

As mentioned above, an augmented reality display device may beconfigured to display virtual information (e.g. computer rendered textand/or imagery) associated with real and/or virtual objects in a useenvironment. However, as an augmented reality display device may have alimited field of view in which augmented reality imagery may bedisplayed, the user may not be aware of objects comprising associatedaugmented reality information that are located outside of this field ofview, such as behind the user. As such, the user may miss out on theopportunity to view and/or interact with the augmented realityinformation for those objects.

Accordingly, embodiments are disclosed herein that relate to providingvia an augmented reality display device indications of augmented realitydata positioned in the environment at locations outside the field ofview available for the presentation of such data. These indications maycomprise any suitable type of indication, including but not limited toaudio indications and/or visual indications.

FIGS. 1A and 1B show an example embodiment of a use environment 100 foran embodiment of an augmented reality display device 104. The depictedaugmented reality display device 104 takes the form of a head-mounteddisplay device worn by a user 106, and comprises a field of view 102 inwhich augmented reality imagery is displayable. The use environment 100takes the form of a street environment comprising a plurality ofintersecting streets and various buildings, including buildings 120,122, 124, 126, and 128. It should be understood that environment 100 ispresented for the purpose of example, and that a use environment maytake any other suitable form.

FIG. 1A shows a perspective of the environment 100 as viewed by user106, and FIG. 1B shows a top view of environment 100. As mentionedabove, the field of view 102 shown in FIGS. 1A and 1B comprises aportion of the environment viewable through the display device 104 thatmay be augmented with virtual imagery. In some embodiments, the field ofview 102 may be substantially coextensive with the user's actual fieldof vision, while in other embodiments the field of view 102 may occupy alesser portion of the user's actual field of vision. The field of view102 comprises a periphery 103 outside of which augmented reality imageryis not displayable by the augmented reality display device 104.

The augmented reality environment 100 may include a variety ofreal-world objects and a variety of virtual objects or visual featuresdisplayable with the real-world surfaces and objects. For example,buildings 120, 122, 124, 128, and 126 may include restaurants. As such,if user 106 requests information about nearby restaurants in environment100, the augmented reality display device may display virtual objectindications of the nearby restaurants in spatial registration with thecorresponding buildings. As a non-limiting example of such anindication, an icon 153 (e.g., a pin or other marking) may be overlaidon building 120 indicating that building 120 includes a restaurant.Additionally, any other suitable information, such as a business name,logo, telephone number, menu, review(s), etc. may be displayed oraccessible via the indication.

However, although buildings 122, 124, 126, and 128 also containrestaurants, user 106 may be unaware of these options due to therestaurants being located outside of the field of view 102. For example,building 122, which includes a restaurant very close to user 106, isjust outside of the field of view 102. As such, the user 106 may not bemade aware of this restaurant unless the user moves to place building122 in the field of view 102.

Thus, various indications, e.g., audio or visual features, may bepresented to user 106 to alert the user to information in theenvironment which is outside of the augmented reality field of view. Theindications may be provided by a user interface that is persistent ornon-persistent (e.g. displayed upon user request), and may provideinformation about available augmented reality information in an extendedsurrounding space. This interface may be visual, e.g., represented byon-screen markers or maps, audible, e.g., where a set of soundsindicates to the user presence and proximity of data, and/or may takeany other suitable form (e.g. tactile/haptic). In this way, a user maybe alerted to digital information embedded in a user's physicalenvironment even when the information is out of sight of the user.

An augmented reality display device according to the present disclosuremay take any suitable form, including but not limited to near-eyedevices such as the head-mounted display device 104 of FIGS. 1A and 1Band other mobile devices. FIG. 2 shows an example embodiment of anaugmented reality display system 300, and FIG. 3 shows a block diagramof the display system 300.

Display system 300 comprises one or more lenses 302 that form a part ofa see-through display subsystem 304, such that images may be displayedvia lenses 302 (e.g. via projection onto lenses 302, waveguide system(s)incorporated into lenses 302, and/or in any other suitable manner).Display system 300 further comprises one or more outward-facing imagesensors 306 configured to acquire images of a background scene and/orphysical space being viewed by a user, and may include one or moremicrophones 308 configured to detect sounds, such as voice commands froma user. Outward-facing image sensors 306 may include one or more depthsensors and/or one or more two-dimensional image sensors. In otherembodiments, an augmented reality display system, instead of asee-through display subsystem, may display augmented reality images viaa viewfinder mode for an outward-facing image sensor.

Display system 300 may further comprise a gaze detection subsystem 310configured to detect a direction of gaze of each eye of a user or adirection or location of focus, as described above. Gaze detectionsubsystem 310 may be configured to determine gaze directions of each ofa user's eyes in any suitable manner. For example, in the depictedembodiment, gaze detection subsystem 310 comprises one or more glintsources 312, such as infrared light sources, configured to cause a glintof light to reflect from each eyeball of a user, and one or more imagesensors 314, such as inward-facing sensors, configured to capture animage of each eyeball of the user. Changes in the glints from the user'seyeballs and/or a location of a user's pupil as determined from imagedata gathered via the image sensor(s) 314 may be used to determine adirection of gaze. Further, a location at which gaze lines projectedfrom the user's eyes intersect the external display may be used todetermine an object at which the user is gazing (e.g. a displayedvirtual object and/or real background object). Gaze detection subsystem310 may have any suitable number and arrangement of light sources andimage sensors. In other embodiments, gaze detection subsystem 310 may beomitted.

Display system 300 may further comprise additional sensors. For example,display system 300 may comprise a global positioning (GPS) subsystem 316to allow a location of the display system 300 to be determined This mayhelp to identify objects, such as buildings, etc., that are located inthe user's surrounding physical environment.

Display system 300 may further include one or more motion sensors 318 todetect movements of a user's head when the user is wearing displaysystem 300. Motion data may be used, potentially along with eye-trackingglint data and outward-facing image data, for gaze detection, as well asfor image stabilization to help correct for blur in images from theoutward-facing image sensor(s) 306. The use of motion data may allowchanges in gaze location to be tracked even if image data fromoutward-facing image sensor(s) 306 cannot be resolved. Likewise, motionsensors 318, as well as microphone(s) 308 and gaze detection subsystem310, also may be employed as user input devices, such that a user mayinteract with the display system 300 via gestures of the eye, neckand/or head, as well as via verbal commands. It will be understood thatsensors illustrated in FIGS. 2 and 3 are shown for the purpose ofexample and are not intended to be limiting in any manner, as any othersuitable sensors and/or combination of sensors may be utilized.

Display system 300 further comprises a controller 320 having a logicsubsystem 322 and a storage subsystem 324 in communication with thesensors, gaze detection subsystem 310, display subsystem 304, and/orother components. Storage subsystem 324 comprises instructions storedthereon that are executable by logic subsystem 322, for example, toreceive and interpret inputs from the sensors, to identify location andmovements of a user, to identify real objects in an augmented realityfield of view and present augmented reality imagery therefore, to detectobjects located outside a field of view of the user, and to presentindications of positional information associated with objects locatedoutside the field of view of the user, among other tasks.

It will be appreciated that the depicted display devices 104 and 300 aredescribed for the purpose of example, and thus are not meant to belimiting. It is to be further understood that the display device mayinclude additional and/or alternative sensors, cameras, microphones,input devices, output devices, etc. than those shown without departingfrom the scope of this disclosure. Additionally, the physicalconfiguration of a display device and its various sensors andsubcomponents may take a variety of different forms without departingfrom the scope of this disclosure.

FIGS. 4-8 show example embodiments of indications of objects outside afield of view 102 available for the display of augmented reality imageryon an augmented reality display device, wherein the indicated objectscorrespond to object having associated augmented reality information.The term “outside of a field of view” and the like as used hereinsignify objects out of an angular field of view, objects occluded byother objects (real or virtual), and/or objects that are not directlyvisible for any other reason. First, FIG. 4 shows an example field ofview 102 at a first time T1 and at a second, later time T2. At time T1,an object 402 and an object 404 are within the field of view 102.Objects 402 and 404 may comprise real or virtual objects, wherein avirtual object may be any object displayed or displayable in anaugmented reality image that is not present in the physical scene andthat is combined with an image of the physical scene to form theaugmented reality image. In this example, a plurality of objects arelocated outside field of view 102. As such, a corresponding plurality ofvisual indications in the form of markers 406, 408, 410, 412, and 414are displayed within field of view 102, thereby indicating the presenceof the objects located outside the field of view 102.

In the depicted embodiment, the markers are displayed adjacent to aperiphery 103 of the field of view 102, but may have any other suitableappearance. Further, the markers may be displayed in a location thatrepresents a relative location of the corresponding out-of-view object.For example, markers 410, 412, and 414 positioned at the right side ofthe field of view 102 may indicate objects positioned to the right ofthe user outside the user's field of view, and thus may indicate adirection to turn to view the corresponding objects. As another example,marker 408 positioned at a top border of the field of view 102 mayindicate that the user can turn or look upwards to view thecorresponding object. As still another example, marker 406 is displayedon the right edge of field of view 102, thereby directing the user tolook or turn to the right to view the corresponding object.

At time T2, the user has shifted their field of view toward the right sothat object 416 associated with marker 412 comes into the user's fieldof view 102. In some examples, marker 412 may continue to be displayedafter object 416 comes into the field of view in order to direct theuser's attention to object 416. However, in other examples, marker 412may no longer be displayed or may change in appearance, e.g., changecolor shape, size, content of information displayed, orientation, etc.,when the corresponding object enters field of view 102. Further, thisshift of the field of view toward the right shown at time T2 causesobject 402 to move outside field of view 102. Thus, a marker 418associated with object 402 is displayed at the left edge of the field ofview to indicate that object 402 is now located outside the field ofview towards the left.

In the depicted example, markers for objects both within and outside offield of view 102 are depicted with a same appearance for clarity.However, it will be understood that different information, or noinformation, may be displayed for objects inside of the field of view.For example, in the example of a search for nearby restaurants, themarkers at the periphery of the field of view 102 may indicate thepresence of out-of-view restaurants, but no name information, whilemarkers for restaurants in the field of view may comprise additionalinformation, such as a restaurant name, phone number, links to menus,reviews, etc. Likewise, in the context of a user search for a specificbusiness, a peripheral marker that indicates the out-of-view location ofthe business may disappear and not be replaced by a different marker, asthe business name may be clearly evident in the real-world environment.

It will be understood that the visual indicators shown in FIG. 4 arepresented for the purpose of example, and that a visual indicator maytake any other suitable form. Further, the indicators may indicateinformation other than relative location. For example, an appearance ofa visual indicator may vary based on properties of the object, such as adistance from the object to the user and/or a direction from the objectto the user. Thus, an appearance of a marker may vary based on changesin the object relative to the user, such as changes in distance fromuser and/or direction relative to user, etc.

Further, the indicators may indicate other information than locationalinformation. For example, an appearance of a visual indicator maysignify density of objects comprising augmented reality information in aparticular direction or location, in that a corresponding number ofmarkers may be displayed on that side of the display. As anotherexample, a size of a visual indicator may be based on a number ofobjects associated with that visual indicator, such that a relativelylarger size of a displayed marker may indicate a relatively largernumber of objects associated with that marker. It will be understoodthat as the user changes perspective or changes the field of view 102 inthe environment, display of the markers may be dynamically updated toreflect changes in features, information, and objects located outsidethe field of view 102.

In some examples, a position at which a visual indicator is displayedmay be determined based on a center point 452 of the field of view 102.For example, the center point 452 of the user's gaze both vertically andhorizontally may be opposite from a dividing line that will determine ifa marker appears on the right, the left, the top, or the bottom alongthe periphery 103 of field of view 102. In such examples, a visibleasset located behind the user and slightly to the right may be indicatedby a marker displayed on the right hand periphery of field of view 102.In other embodiments, visual indicators may be positioned in any othersuitable manner.

FIG. 5 shows another example visual indicator in the form of a virtualmap 506 of augmented reality assets. At time T1, object 502 and object504 are within the field of view 102 of the user. The virtual map 506represents a physical area in the environment greater than the field ofview 102. For example, the field of view may be indicated as rectangularregion with a width 532 containing a center region 508 less than thewidth 532 of the virtual map, wherein the center region 508 representsthe field of view 102 of the user. In some examples, center region 508may include a center indicator or dot 510 which represents the center512 of field of view 102.

The center region 508 may assist the user in identifying elements orobjects having associated augmented reality information which aredisplayed within field of view 102 and those that are displayed outsidefield of view 102. For example, as shown at time T1, marker 502 adisplayed within center region 508 may correspond to object 502. Markersfor objects outside field of view 102 are displayed outside of centerregion 508 within virtual map 506. For example, visual indicators ormarkers 518, 516 a, 524, and 526 are displayed on virtual map 506 toindicate the presence of objects located outside the field of view 102.

The virtual map 506 may be displayed in any suitable manner. Forexample, the virtual map 506 may be displayed as a persistent headlocked map that shows a layout of elements around the user, whereinmarkers displayed on the map may change location with user movements.Further, in some embodiments, the appearance of markers displayed in thevirtual map may be based on the type of objects which they represent.For example, a marker in virtual map 506 with a first color may indicatea first type of object (e.g. a restaurant) whereas a marker in virtualmap 506 with a second color may represent a second type of object (e.g.a retail store).

The virtual map 506 may be displayed at any suitable location on adisplay and may have any suitable appearance. For example, in thedepicted embodiment, the virtual map 506 is displayed at a bottom regionof field of view 102. Further, the virtual map 506 may be at leastpartially translucent or transparent so that the virtual map 506 doesnot intrude on the user's view of elements in field of view 102. In thedepicted embodiment, the virtual map may display representations ofobjects within a 360 degree area surrounding the user, but may indicateany other suitable angular range.

Continuing with FIG. 5, at time T2, the user's field of view has shiftedtoward the right so that an object 516 associated with marker 516 acomes into the user's field of view 102. As shown at time T2, a marker516 b is now displayed in center region 508 indicating that object 516is now located within field of view 102. In some examples, an appearanceof a marker associated with an object displayed on virtual map 506 maychange depending on whether or not the object is in the field of view102. For example, marker 522 may be displayed in a first manner whenobject 516 is located outside the field of view 102, e.g., with a firstcolor, size, and/or shape, and may be displayed in a second differentmanner when object 516 enters field of view 102. It will be noted thatthis shift of the field of view toward the right shown at time T2 causesobject 502 to move outside field of view 102. Thus, marker 520 bassociated with object 502 is displayed on virtual map 506 outside ofcentral region 508 toward the left of the central region to indicatethat that object 502 is now located outside the field of view towardsthe left.

FIG. 6 shows another example embodiment of a visual indicator. In thisexample, a room environment 602 is shown in field of view 102. At timeT1, a visual indicator 604 is displayed at a bottom right periphery offield of view 102 to indicate the presence of an object located outsidefield of view 102 in a direction outside the bottom right of field ofview 102. The visual indicator 604 may comprise a route, a path, anarrow, or other indication of directionality which a user may follow inorder to be directed to the object it represents. In the depictedembodiment, visual indicator 604 may be represented as a tendril (e.g. avine-like feature or other representation of a line) extending fromobject 606, which may represent a notification block or other virtualobject. Thus, a user may visually follow indicator 604 towards object606 by turning towards the right to bring the object 606 into view.

A tendril may be used in any suitable context to lead a user's gaze to aobject located outside of a current augmented reality field of view. Thetendril acts as a line that leads from within the field of view towardthe object outside the field of view. Thus, a user may visually followthe tendril to find the object to which the tendril leads. Such tendrilsmay be displayed persistently for out-of-view objects, or may bedisplayed upon occurrence of a suitable trigger. For example, a tendrilfor an out-of-view object may appear upon receipt of a user input(voice, gaze, gesture, etc.) to select or view the object. In otherexamples, a tendril may be displayed in response to a world recognitionevent (e.g. recognizing an object, event, or other real-world item ofinterest).

As another example, display of a tendril may be triggered when acorresponding out-of-view object has new activity to be viewed. Forexample, a tendril may be displayed to lead a viewer's gaze to anout-of-view email inbox when a new email message is received.Additionally, a tendril may be used to indicate an out-of-view objectthat currently has user focus. For example, if a user is currentlyentering text into a text entry box and then gazes in a differentdirection such that the text box is no longer visible, the display of atendril that leads back to the text box may be triggered. A tendril alsomay lead a user to objects that are obscured by other objects. Forexample, a tendril may lead around a corner or a wall to an out-of-viewobject. In any of these examples, a tendril may persist or disappearwhen the out-of-view object comes into view.

A tendril may have any suitable appearance. For example, in someembodiments a tendril may have a vine-like appearance. In otherembodiments, the tendril may have the appearance of any other suitablerepresentation of a path leading to an out-of-view object. A tendril mayoriginate from a location adjacent to a periphery of a display, or mayoriginate closer to a center of a user's field of view. Further, asdescribed above with regard to other embodiments, a color, shape, size,opacity, and/or other suitable property of a tendril may be varied basedupon object properties such as object type, location, and/or virtual orreal world placement.

In some embodiments, audio cues may be used to alert a user to thecreation of a tendril. For example, using the email example above, anaudio output, such as a hum, tone, or other suitable sound, may beoutput to indicate the creation of a tendril upon receipt of a new emailmessage in an out-of-view email inbox. Further, a characteristic of thesound may be varied to provide additional information on the object. Forexample, a high pitched hum may be used to indicate that the object isfar away, while a low pitched hum may indicate an object that is nearby.It will be understood that these embodiments are described for thepurpose of example, and are not intended to be limiting in any manner.

In some examples, a visual indicator may represent an object that ismoved from within the field of view available for the display ofaugmented reality imagery. For example, referring to FIG. 7, a user inan environment 702 may initially be interacting with an object 704, suchas a photograph, user interface control, application interface/window,or any other suitable object, as shown at time T1.

When the user has finished interacting with the object 704, the objectmay be moved to a location outside of field of view 102 (automaticallyor by user action). Thus, as illustrated at time T2, an animation 706may be displayed within field of view 102 indicating movement of object704 toward the out-of-view location. The appearance of animation 706 maybe based on various properties of the movement of object 704 and/or alocation to which object 704 is sent. For example, a velocity of theanimation 706 may indicate how far off screen the object 704 is going toland, wherein a relatively higher velocity may indicate a relativelygreater distance outside the field of view. Animation 706 may take anyother suitable form, including but not limited to pulses extending alonga route or tendril that indicates the location to which object 704 ismoving. Next referring to time T3, after the object 704 moves outsidefield of view 102, a marker 708 may be displayed at periphery 103 offield of view 102 indicating the location at which the object 704 ispositioned outside the field of view 102.

Further, in some embodiments, an animation may be displayed within fieldof view 102 which indicates actions performed on an object locatedoutside the field of view. For example, changes in an appearance of atendril based upon user interaction with an object to which the tendrilleads may be animated to indicate user activity with the object. As amore specific example, if a user is looking at an object, e.g., a textentry box, displayed in the field of view and provides input via areal-world input device, e.g., a keyboard, located outside the field ofview, then an animation or other suitable indicator may be displayed asoriginating at the real-world input device and terminating at the textentry box within the field of view to indicate this user activity withthe out-of-view object (the keyboard). Such an animation may have anysuitable appearance. As one non-limiting example, as the user providesinput to the real-world input device, e.g., as the user presses the keyson a keyboard located outside the field of view, an animated pulsetraveling up a tendril from the input device to the text entry box maybe displayed.

FIG. 8 shows another example use of tendrils to indicate positionalinformation associated with out-of-view objects in an augmented realityenvironment 802. More specifically, FIG. 8 shows a plurality of virtualobjects 818, 820, 822 and 824 located outside a field of view 806 of anaugmented reality display device. Virtual objects 818, 820, 822 and 824may be application user interfaces or any other suitable object withwhich a user of the display device may interact. FIG. 8 also shows agroup of indicators 810, 812, 814, and 816, such as icons, associatedwith objects 818, 820, 822 and 824, wherein the indicators are currentlydisplayed at a position that is currently within field of view 806. Forexample, indicator 810 may correspond to object 824, indicator 812, maycorrespond to object 834, indicator 814 may correspond to object 820,and indicator 816 may correspond to object 818. FIG. 8 also depicts areal object in the form of a keyboard 808.

Virtual objects 818, 820, 822, and 824 may be “world locked,” in thatthey may not move relative to a real-world background as user'sperspective changes in the environment 802. As such, a virtual objectmay at times be out of the field of view 806. Thus, a user may locateobjects by viewing the group of indicators and selecting an indicatorcorresponding to the desired virtual object (e.g. by selecting an iconcorresponding to a desired application). The selection may be made byvoice, gaze, body gesture (e.g. head gesture as detected by motionsensors, arm gesture as detected by an image sensor, etc.), and/or anyother suitable mechanism. This selection may result in the display of atendril 826 in field of view 806 to lead the user's gaze to theassociated object. In FIG. 8, the part of the tendril 826 that iscurrently in a user's field of view is shown in solid lines, and thepart that is currently out of the field of view is shown in dashedlines. By visually following the tendril for its length, the user maylocate the desired virtual object.

FIG. 9 shows a flow diagram depicting an example of a method 900 foroperating a user interface on an augmented reality computing device toprovide indications of real and/or virtual objects located outside afield of view of a user. As described below, these indications mayinclude visual indications and/or audio indications.

At 902, method 900 includes identifying objects outside a field of viewof a user that include associated augmented reality information.Identifying objects outside the field of view includes obtaininginformation about the objects in any way, e.g., information generated ordetected locally and/or information obtained from other devices. Theobjects may comprise any suitable real and/or virtual objects. Method900 further comprises indicating positional information associated withthe objects outside of the field of view of the user. The indication maycomprise information regarding a position of the object relative to aposition of the user to convey to the user a direction to look to viewthe objects, and/or any other suitable information regarding theobjects.

The indication may be provided in response to any suitable trigger. Insome examples, the indications may be provided to the user in responseto a user input, e.g., a gesture (gaze gesture, body gesture, etc.), aspeech input, or other suitable input. In other examples, indicationsmay be automatically provided to the user based upon a programmatictrigger, which may depend on the user's environment.

As mentioned above and shown at 906, presenting the indicator maycomprise displaying a visual indicator via a display device. Thedisplayed visual indicator may have any suitable appearance. Forexample, as shown at 908, the visual indicators may take the form ofmarkers. Further, an appearance of the markers may indicate informationsuch as a direction to turn to view the object, a distance of the objectfrom the field of view available for the display of augmented realityimagery, etc. The markers may be displayed at the periphery of the fieldof view, or at any other suitable location.

As another example, visual indicators also may take the form of paths,such as tendrils, that a user can visually follow to find the associatedobjects, as indicated at 910. For example, a marker may comprise avisually displayed path extending from a location within the field ofview of the user towards or to the object. As still another example,visual indicators may be displayed via a virtual map, as indicated at912. Such a map may take the form of an overlay that maps a physicalarea larger than the field of view useable to display augmented realityimagery, and may have any suitable appearance, non-limiting examples ofwhich are described above.

As a user turns or moves in response to the display of visualindicators, the display of the visual indicators may change, asindicated at 914. For example, an appearance of a marker may be variedbased on one or more of a property of the object, a distance from theobject to the user, and an orientation of the object relative to theuser. Varying a display of a marker may comprise one or more of varyinga color of the object, varying an opacity of the object, varying a sizeof the object, varying a shape of the object, and varying a movement ofthe object. Where the appearance of a marker is varied based upon aproperty of the object, the property may comprise a global property or acontext-specific (e.g. application specific) property. For example, inan augmented reality video game, a color of a marker may be used toindicate a relative value/importance of a game object (e.g. a highestvalue or most difficult enemy).

Further, the indication of the objects out of the user's view also maycomprise audio indications, instead of or in addition to visualindicators. Thus, at 916, method 900 may include emitting one or moresounds from speakers indicating a presence of an object or objects. Theaudio indications may take any suitable form. For example, theindications may take the form of chimes/bells, beeps, other tones, aswell as more complex outputs, such as computerized speech outputs.

As with the visual indicators, the presentation of an audio indicationmay change as the user's position changes relative to out-of-viewobjects, or based upon other state changes, as indicated at 918. Thesounds may be varied based on any suitable factors, such as one or moreof a property of the object, a distance from the object to the user,and/or an orientation of the object relative to the user. Further, thesounds may be varied in any suitable manner. For example, the sounds maybe varied by varying a volume of the one or more sounds, varying one ormore frequencies of the one or more sounds, varying a timbre of thesound, varying an apparent position of the one or more sounds (e.g. byapplying a head-related transfer function (HRTF) based upon the relativelocations of an object and a user), and varying a tactile/hapticresponse. As a more specific example, a user may request “are there anycoffee shops nearby” via speech or text input. In response, the relativelocations of coffee shops may be represented by sequentially presentedoutput tones (one for each coffee shop), wherein the signals applied toeach of two or more speakers (e.g. earphones on a head-mounted displaydevice) for each tone are modified based upon an HRTF determined basedupon the relative locations of the user and the associated coffee shop.

In other examples, the distance to the object and/or relative locationof the object may also be indicated either by sound pattern, by tempo,by pitch, or by length of the sound. Such sound qualities may further beused to indicate the density of the available information, and/or anyother suitable information. For example, a density of objects on eachside of a display may be indicated by relative differences in pitch ofsounds used to indicate the densities.

In some embodiments, the methods and processes described above may betied to a computing system of one or more computing devices. Inparticular, such methods and processes may be implemented as acomputer-application program or service, an application-programminginterface (API), a library, and/or other computer-program product.

FIG. 10 schematically shows a non-limiting embodiment of a computingsystem 1000 that can enact one or more of the methods and processesdescribed above. Display device 104 may be one non-limiting example ofcomputing system 1000. Computing system 1000 is shown in simplifiedform. It will be understood that virtually any computer architecture maybe used without departing from the scope of this disclosure. Indifferent embodiments, computing system 1000 may take the form of adisplay device, wearable computing device, mainframe computer, servercomputer, desktop computer, laptop computer, tablet computer,home-entertainment computer, network computing device, gaming device,mobile computing device, mobile communication device (e.g., smartphone), etc.

Computing system 1000 includes a logic subsystem 1002 and a storagesubsystem 1004. Computing system 1000 may optionally include a displaysubsystem 1006, input subsystem 1008, communication subsystem 1010,and/or other components not shown in FIG. 10.

Logic subsystem 1002 includes one or more physical devices configured toexecute instructions. For example, logic subsystem 1002 may beconfigured to execute instructions that are part of one or moreapplications, services, programs, routines, libraries, objects,components, data structures, or other logical constructs. Suchinstructions may be implemented to perform a task, implement a datatype, transform the state of one or more components, or otherwise arriveat a desired result.

Logic subsystem 1002 may include one or more processors configured toexecute software instructions. Additionally or alternatively, logicsubsystem 1002 may include one or more hardware or firmware logicmachines configured to execute hardware or firmware instructions. Theprocessors of logic subsystem 1002 may be single-core or multi-core, andthe programs executed thereon may be configured for sequential, parallelor distributed processing. Logic subsystem 1002 may optionally includeindividual components that are distributed among two or more devices,which can be remotely located and/or configured for coordinatedprocessing. Aspects of logic subsystem 1002 may be virtualized andexecuted by remotely accessible, networked computing devices configuredin a cloud-computing configuration.

Storage subsystem 1004 includes one or more physical devices configuredto hold data and/or instructions executable by logic subsystem 1002 toimplement the methods and processes described herein. When such methodsand processes are implemented, the state of storage subsystem 1004 maybe transformed—e.g., to hold different data.

Storage subsystem 1004 may include removable media and/or built-indevices. Storage subsystem 1004 may include optical memory devices(e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memorydevices (e.g., RAM,

EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g., hard-diskdrive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storagesubsystem 904 may include volatile, nonvolatile, dynamic, static,read/write, read-only, random-access, sequential-access,location-addressable, file-addressable, and/or content-addressabledevices.

It will be appreciated that storage subsystem 1004 includes one or morephysical device, and excludes propagating signals per se. However, insome embodiments, aspects of the instructions described herein may bepropagated by a pure signal (e.g., an electromagnetic signal, an opticalsignal, etc.) via a communications medium, as opposed to being stored ona storage device. Furthermore, data and/or other forms of informationpertaining to the present disclosure may be propagated by a pure signal.

In some embodiments, aspects of logic subsystem 1002 and of storagesubsystem 1004 may be integrated together into one or morehardware-logic components through which the functionally describedherein may be enacted. Such hardware-logic components may includefield-programmable gate arrays (FPGAs), program- andapplication-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC) systems, and complex programmable logic devices (CPLDs), forexample.

When included, display subsystem 1006 may be used to present a visualrepresentation of data held by storage subsystem 1004. This visualrepresentation may take the form of a graphical user interface (GUI). Asthe herein described methods and processes change the data held by thestorage subsystem, and thus transform the state of the storagesubsystem, the state of display subsystem 1006 may likewise betransformed to visually represent changes in the underlying data.Display subsystem 1006 may include one or more display devices utilizingvirtually any type of technology. Such display devices may be combinedwith logic subsystem 1002 and/or storage subsystem 1004 in a sharedenclosure, or such display devices may be peripheral display devices.

When included, input subsystem 1008 may comprise or interface with oneor more user-input devices such as a keyboard, mouse, touch screen, orgame controller. In some embodiments, the input subsystem may compriseor interface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity.

When included, communication subsystem 1010 may be configured tocommunicatively couple computing system 1000 with one or more othercomputing devices. Communication subsystem 1010 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. As non-limiting examples, the communicationsubsystem may be configured for communication via a wireless telephonenetwork, or a wired or wireless local- or wide-area network. In someembodiments, the communication subsystem may allow computing system 1000to send and/or receive messages to and/or from other devices via anetwork such as the Internet.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnon-obvious combinations and sub-combinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof

1. An augmented reality computing device comprising: a display system; alogic device; and a storage device comprising instructions executable bythe logic device to identify one or more virtual objects located outsidea field of view available for a display of augmented realityinformation, and for each object of the one or more virtual objects,provide to the user an indication of positional information associatedwith the virtual object within the field of view adjacent to a peripheryof the field of view.
 2. The augmented reality computing device of claim1, wherein the indication of positional information associated with thevirtual object includes an indication of a position of the virtualobject relative to a position of the user.
 3. The augmented realitycomputing device of claim 2, wherein the instructions are executable todisplay the indication positional information associated with thevirtual object on the display system.
 4. The augmented reality computingdevice of claim 2, wherein the instructions are executable to displaythe indication positional information associated with the virtual objecton the display system within the field of view of a user as a markerindicating a presence of the virtual object and a direction to turn toview the virtual object.
 5. The augmented reality computing device ofclaim 4, wherein the instructions are executable to display the markeras a tendril extending from a location within the field of view to thevirtual object.
 6. The augmented reality computing device of claim 4,wherein the instructions are further executable to vary a display of themarker based on one or more of a property of the virtual object, adistance from the virtual object to the user, and an orientation of thevirtual object relative to the user.
 7. The augmented reality computingdevice of claim 6, wherein the instructions are executable to vary thedisplay of the marker by one or more of varying a color of the marker,varying an opacity of the marker, varying a size of the marker, varyinga shape of the marker, and varying a movement of the marker.
 8. Theaugmented reality computing device of claim 1, wherein the instructionsare executable to provide the indication of positional informationassociated with the virtual object as one or more sounds emitted fromspeakers indicating a presence of the virtual object.
 9. The augmentedreality computing device of claim 8, wherein the instructions arefurther executable to vary the one or more sounds emitted from speakersbased on one or more of a property of the virtual object, a density ofobjects, a distance from the virtual object to the user, and anorientation of the virtual object relative to the user.
 10. Theaugmented reality computing device of claim 9, wherein the instructionsare executable to vary the one or more sounds emitted from speakers byone or more of varying a volume of the one or more sounds, varying oneor more frequencies of the one or more sounds, varying a position of theone or more sounds, and varying a tactile response.
 11. The augmentedreality computing device of claim 1, wherein the display systemcomprises a see-through display system, and wherein the instructions areexecutable to display the indication of positional informationassociated with the virtual object on the see-through display system asa virtual map.
 12. The augmented reality computing device of claim 1,wherein the instructions are executable to provide the indication ofpositional information associated with the virtual object in response touser input.
 13. An augmented reality computing device comprising: asee-through display system, a logic device; and a storage devicecomprising instructions executable by the logic device to identify anobject located outside a field of view of the see-through display systemavailable for a display of augmented reality information; and displayvia the see-through display system a tendril that leads from a locationwithin the field of view toward the object, the tendril indicating avisual path to follow to locate the object.
 14. The augmented realitycomputing device of claim 13, wherein instructions are executable todisplay the tendril as extending to the object.
 15. The augmentedreality computing device of claim 13, wherein the instructions arefurther executable to animate an appearance of the tendril based uponuser interaction with the object.
 16. The augmented reality computingdevice of claim 13, wherein the instructions are further executable toemit a sound from one or more speakers to indicate creation of thetendril.
 17. The augmented reality computing device of claim 13, whereinthe instructions are further executable to cease displaying the tendrilwhen the object enters the field of view.
 18. The augmented realitycomputing device of claim 17, wherein the instructions are furtherexecutable to vary a display of the tendril based on one or more of aproperty of the object, a distance from the object to the user, and anorientation of the object relative to the user, and wherein theinstructions are executable to vary a display of the tendril by one ormore of varying a color of the tendril, varying an opacity of thetendril, varying a size of the tendril, varying a shape of the tendril,and varying a movement of the tendril.
 19. On a see-through displaydevice, a method comprising: detecting one or more objects locatedoutside a field of view of the see-through display device available forthe display of augmented reality imagery; for each object of the one ormore objects, providing indications to the user of presence anddirection of the object located outside the field of view, theindications including a virtual map representing a physical area greaterthan the field of view in a vertical user gaze direction and ahorizontal user gaze direction; and varying the indications provided tothe user based on one or more of a change in a location of the objectrelative to the see-through display device, a distance of the objectfrom the see-through display device, and a property of the object. 20.The method of claim 19, wherein the indications comprise one or more ofmarkers displayed on the see-through display device and sounds emittedfrom speakers.